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One  of  the  most  ambitious  efforts  in  value-centric  design  of  a  military  aerospace  system 
undertaken  to  date  has  been  the  parallel  development  by  four  performer  teams,  headlined 
by  major  space  industry  primes,  of  design  tools  for  fractionated  space  architectures  under 
DARPA's  System  F6  program.  The  goal  of  the  System  F6  program  is  to  replace  traditional, 
highly-integrated,  monolithic  satellites  with  wirelessly-networked  clusters  of  heterogeneous 
modules  incorporating  the  various  payload  and  infrastructure  functions.  Such  fractionated 
architectures  can  deliver  a  comparable  or  greater  mission  capability  than  monolithic 
satellites,  but  with  significantly  enhanced  flexibility  and  robustness.  In  order  to  design  an 
optimal  fractionated  architecture,  the  potential  cost  penalties  due  to  the  overhead  of  such  a 
design  must  be  balanced  against  the  value  enhancement  due  to  improved  flexibility  and 
robustness. 

The  first,  preliminary  design  phase  of  the  System  F6  program,  simultaneously  awarded 
to  four  competing  industry  teams  led  by  Boeing,  Lockheed  Martin,  Northrop  Grumman, 
and  Orbital  Sciences,  commenced  in  February  2008  and  included  a  significant  effort  for  the 
development,  validation,  and  demonstration  of  a  Value-Centric  Design  methodology  and 
associated  tool  suite  that  can  support  the  design  of  optimized  fractionated  satellite  systems 
based  on  a  net  lifecycle  value  metric  and  a  probabilistic  distribution  thereof.  This  phase 
concluded  in  February  2009  and  the  Value-Centric  Design  methodology  development  to  date 
is  documented  in  a  series  of  papers  by  the  industry  performer  teams.  This  paper,  from  the 
System  F6  Program  Office,  summarizes  the  overarching  objectives  of  the  Value-Centric 
Design  effort,  details  and  rationalizes  the  requirements  for  the  methodology,  discusses  the 
relationship  between  Value-Centric  Design  and  the  traditional  industry-standard  systems 
engineering  process,  and  fills  any  gaps  in  the  performers'  own  presentations  of  their  efforts, 
tools,  and  results. 


I.  Introduction 


The  DARPA  E6  Program  is  an  exploration  and  demonstration  of  fractionated  spacecraft,  where  a  wirelessly 
linked,  free-flying  cluster  of  spacecraft  acts  as  a  single  system.  In  the  simplest  and  most  obvious  instances  of 


*  The  views  expressed  herein  are  the  authors’  own  and  do  not  necessarily  represent  those  of  the  Defense  Advanced 
Research  Projects  Agency,  the  Department  of  Defense,  or  the  United  States  Government.  Approved  for  public 
release.  Distribution  unlimited. 

'  Program  Manager,  Tactical  Technology  Office,  owen.brown@darpa.mil,  AIAA  Senior  Member. 

'  Program  Manager,  Tactical  Technology  Office,  paul.eremenko@darpa.mil,  AIAA  Senior  Member. 

Executive  Director,  paul.collopv@vddi.org,  AIAA  Associate  Eellow. 

1 

American  Institute  of  Aeronautics  and  Astronautics 


This  material  is  declared  a  work  of  the  U.S.  Government  and  is  not  subject  to  copyright  protection  in  the  United  States. 


Report  Documentation  Page 


Form  Approved 
0MB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 


1.  REPORT  DATE 

SEP  2009 

2.  REPORT  TYPE 

3.  DATES  COVERED 

00-00-2009  to  00-00-2009 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Value-Centric  Design  Methodologies  for  Fractionated  Spacecraft: 

Progress  Summary  from  Phase  1  of  the  DARPA  System  F6  Programl 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Defense  Advanced  Research  Projects  Agency, Arlington, VA, 22203 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distrihution  unlimited 

13.  SUPPLEMENTARY  NOTES 

AIAA  SPACE  2009  Conference  &  Exposition,  14-17  Sep  2009,  Pasadena,  CA 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

1 

1  ^ 

16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

unclassified 


b.  ABSTRACT 

unclassified 


c.  THIS  PAGE 

unclassified 


17.  LIMITATION  OF 
ABSTRACT 

Same  as 
Report  (SAR) 


18.  NUMBER 
OF  PAGES 

14 


19a.  NAME  OF 
RESPONSIBLE  PERSON 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


fractionation,  a  cluster  could  functionally  replace  a  single  traditional  monolithic  satellite.  Perhaps  an  earth- 
observing  weather  satellite  is  replaced  by  an  imaging  module  (a  “module”  being  synonymous  with  an  individual 
spacecraft),  a  computation  and  data  handling  module,  and  a  communications  module,  all  physically  separate  but 
connected  by  wireless  data  links.  More  advanced  implementations  of  fractionation  would  be  capable  of 
functionality  beyond  the  reach  of  a  single  traditional  spacecraft  (by  creating  sparse  apertures,  for  example).  Such 
alternative  capability,  though  very  possible,  is  not  the  primary  motivation  for  fractionation.  As  will  be  discussed 
below,  breaking  out  of  the  conceptual  bond  that  ties  a  space  system  to  a  physical  spacecraft  opens  the  door  for  more 
flexible,  robust,  responsive,  and  ultimately  more  cost  effective  space  systems. 

The  F6  Program  plans  to  build  and  fly  a  fractionated  spacecraft  system  to  demonstrate  the  concept  and  the 
potential  value  of  fractionation,  enabling  rapid  transition  of  this  technology  to  existing  needs.  Phase  1  of  the 
program,  completed  in  February,  2009,  funded  four  performer  teams  to  perform  conceptual  design  and  much  of  the 
preliminary  design  for  different  fractionated  spacecraft  systems.  Boeing,  Lockheed  Martin,  Northrop  Grumman, 
and  Orbital  Sciences  each  led  one  of  the  teams. 

These  teams  were  provided  with  very  few  specific  mission  requirements.  Instead  performance  objectives — 
applicable  to  a  wide  variety  of  space  missions  and  stakeholders — served  as  the  driving  basis  of  the  architectural 
approach  .  The  teams  then  chose  reference  missions  on  their  own  and  applied  a  novel  design  approach  called  Value- 
Centric  Design  ,  which  bases  system  engineering  trade  decisions  on  the  net  present  value  (NPV)  and  the  variance  of 
NPV  for  potential  architectural  solutions. 

Each  of  the  performer  teams  has  published  their  approach  to  and  methodology  for  Value-Centric  Design. 

This  paper  will  summarize  the  lessons  learned  by  the  performers  and  add  the  perspective  of  the  government  team. 

A.  Key  Concepts  from  Previous  Work 

We  have  written  before  on  Value-Centric  Design. For  convenience,  we  will  recapitulate  here  some  of  the 
key  ideas  from  those  earlier  papers. 

7.  Requirements,  Capability,  and  Cost 

A  focus  on  achieving  capabilities  embodied  in  requirements  while  minimizing  cost  has,  under  the  influence  of 
technical  and  programmatic  uncertainty,  led  to  ever  more  complex  spacecraft  with  higher  and  higher  cost,  a  “cost- 
complexity  death  spiral.”^  Decision  makers  respond  to  increased  marginal  cost  by  increasing  the  scale  of  spacecraft 
to  maximize  the  overall  capability/cost  quotient,  and  increasing  lifetime  to  minimize  amortized  annual  costs. ^  Both 
trends  increase  capability,  which  drives  further  increases  in  scale  and  lifetime.  The  result  is  very  large,  very 
complex,  and  very  costly  monolithic  spacecraft.  Although  much  of  the  complexity  in  the  design  of  monolithic 
spacecraft  is  intended  to  address  uncertainty  (for  example,  building  in  tolerance  to  space  environments,  with  added 
margin)  complexity  itself  makes  systems  more  fragile,  in  the  sense  that  they  are  vulnerable  to  many  more 
unmodeled  and  unanticipated  failure  modes.  Often  such  failures  may  be  manifested  as  unplanned  programmatic 
problems,  not  just  operational  failures  in  space. 

Escape  from  the  death  spiral  necessitates  breaking  out  of  the  requirements-centric  mindset.  Value-Centric 
Design  moves  the  focus  away  from  just  requirements  toward  finding  a  balance  between  cost  and  value,  while  also 
accounting  for  the  variance  of  each  in  a  given  architecture.  “Value  is  a  measure — wholly  apart  from  cost — that 
reflects  the  utility  of  a  particular  system  to  its  owner  or  operator.”*  Early  work  on  Value-Centric  Design  developed 
the  utility  of  flexibility  and  robustness,  and  responsiveness.  These  are  all  attributes  that  are  of  great  concern  to 
operators  but  tend  to  be  underemphasized  in  requirements-driven  programs  because  their  value  is  realized  in  the  face 
of  uncertainties,  such  as  program  delays,  funding  reductions,  launch  failures,  and  on-orbit  and  on  the  ground  events. 
Requirements  are  generally  built  around  a  scenario,  or  cast  of  scenarios,  that  envision  what  is  predicted  to  happen. 
A  system  that  is  built  to  adapt  to  new  (but  not  yet  fully  vetted)  requirements,  or  is  less  likely  to  experience  cost 
growth  because  of  undesirable  circumstances,  is  not  afforded  a  quantifiable  measure  of  goodness  that  can  be 
compared  with  the  less  adaptable  or  less  robust  alternative.  Value-Centric  Design  offers  a  toolset  to  allow  such 
quantitative  comparisons  to  be  made. 

2.  Key  Definitions 

Value-Centric  Design.  The  incorporation  of  value  metrics,  in  particular  net  value  and  the  variance  in  net 
value,  into  Systems  Engineering.* 

Flexibility.  The  ability  of  a  system  to  change  on  demand.  This  incorporates  scalability,  evolvability, 
maintainability,  and  adaptability.* 

Robustness.  The  intrinsic  ability  of  a  system  to  maintain  functionality  in  response  to  unforeseen 
circumstances.  This  incorporates  reliability,  survivability,  resistance  to  fragility,  and  fault  tolerance.* 


E6  is  an  acronym  for  Euture  East,  Elexible,  Eractionated,  Eree-Elying  Spacecraft  united  by  Information  eXchange. 
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Responsive  Space.  The  capability  of  space  systems  to  respond  rapidly  to  uncertainties,  including  technical 
uncertainty,  environmental  uncertainty,  launch  uncertainty,  demand  uncertainty,  requirements 
uncertainty  and  funding  uncertainty.^ 

5.  Risk  Management 

Value-Centric  Design  fundamentally  improves  the  systems  engineering  risk  management  process  by  allowing  it 
to  become  entirely  quantitative.  Rather  than  minimizing  risk,  Value-Centric  Design  will  locate  a  Pareto  frontier  of 
maximized  risk-adjusted  net  values  for  a  collection  of  possible  investments.  In  other  words,  for  a  given  cost,  it  will 
show  the  architecture  with  the  maximum  net  present  value.  Likewise,  for  each  architectural  choice,  Value-Centric 
Design  will  also  inform  the  decision  maker  of  the  expected  variance  in  value  and  cost,  based  on  the  forecast 
uncertainty  in  outcomes  for  key  life  cycle  events.  Risk  mitigation  is  no  longer  a  process  divorced  from  the  system 
design  trade  process.  Instead,  mitigation  strategies  are  selected  to  trade  net  value  against  reductions  in  the  variance 
of  net  value.* 

4.  Acquisition 

With  Value-Based  Acquisition,  acquisition  decisions  maximize  net  value  for  a  given  cost.  Previous  attempts  at 
Value-Based  Acquisition  have  foundered  on  the  incommensurability  of  derived  performance  parameters  such  as 
flexibility,  but  Value-Centric  Design  promises  to  bring  cost  together  with  performance  attributes  and  the  less 
tangible  derived  attributes  (to  also  include  robustness,  and  responsiveness)  in  a  system  value  model  that  links  all  the 
attributes  to  net  value. 

B.  Overview 

Section  II  below  lays  groundwork  with  a  fundamental  theoretical  presentation  of  Value-Centric  Design.  Section 
III  emphasizes  applications  of  Value-Centric  Design  in  the  conceptual  and  preliminary  design  stages  of  system 
development,  informed  by  experience  from  Phase  1  of  F6.  Section  IV  concentrates  on  risk  management,  reviewing 
results  from  the  early  F6  work  and  discussing  where  this  might  lead.  Section  V  looks  forward  to  the  detailed  design 
stage  and  suggests  opportunities  for  improving  the  Systems  Engineering  process  through  application  of  Value- 
Centric  Design.  Section  VI  explores  the  topic  of  Value-Centric  Acquisition,  updating  the  perspective  of  our  earlier 
papers.  Section  VII  specifically  addresses  the  F6  program  and  the  impact  that  Value-Centric  Design  is  making  on 
the  understanding  and  demonstration  of  fractionated  spacecraft. 

II.  The  Essence  of  Value-Centric  Design 

This  section  identifies  the  fundamentally  distinctive  aspects  of  Value-Centric  Design,  which  are  contrasted  with 
the  requirements-centric  status  quo. 

A.  Decisions  and  Outcomes 

Value-Centric  Design  addresses  the  decision-making  elements  of  design.'  Decisions  are  evaluated  prospectively, 
that  is,  according  to  their  anticipated  outcomes.  In  design,  the  outcome  is  the  life  cycle  value  and  cost  of  the  system, 
from  manufacturing  through  operation  until  retirement.  Under  Value-Centric  Design,  design  decisions  strive  to 
choose  the  desired  system.  The  “desired”  system  is  the  one  that  the  stakeholder  desires,  which  necessitates  a  trade 
between  value  and  cost,  and  the  variance  in  each.  A  system  value  model  estimates  these  value,  cost,  and  variance 
metrics  for  a  variety  of  architectures.  The  value  model  assigns  a  cost  to  a  particular  prospective  system  based  on  its 
anticipated  attributes,  including  design  cost,  manufacturing  cost,  and  launch  cost.  Value  is  determined  based  on 
payload  performance  capability,  availability,  and  the  user’s  demand  for  service. 

Design  is  inherently  uncertain,  even  moreso  in  the  conceptual  and  preliminary  stages.  If  we  knew  exactly  the 
outcome  of  each  design  choice,  design  would  be  nothing  more  than  cranking  out  product  definitions.  Instead, 
designers  face  decisions  with  uncertain  outcomes,  so  that  the  attributes  of  a  prospective  system  design  are  best 
described  by  random  variables.  Therefore,  in  order  to  determine  that  one  design  alternative  is  better  than  another, 
we  adhere  to  the  logic  of  decision  making  under  uncertainty,  which  tells  us  that  the  best  design  is  one  with  the 
highest  expected  utility.*^  We  can  build  this  logic  into  the  system  value  model.  When  we  add  in  adjustments  for 
value  over  time  (cashflow  discounting^^),  and  subtraction  of  costs  from  benefits,  the  result  is  a  system  value  model 
that  produces  a  risk-adjusted  expectation  of  net  present  value  based  on  the  probability  distribution  of  design 
outcomes,  described  as  a  set  of  points  in  a  Euclidean  space  of  design  attributes.  We  generally  choose  the  design 
with  the  highest  value,  so  defined. 


Hazelrigg*°  provides  an  excellent  discussion  of  design  as  a  decision-making  process. 
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B.  Value-Centric  Design  Compared  with  Requirements-Centric  Design 

In  traditional  requirements-driven  systems  engineering  process,*  design  choices  are  based  on  whether  or  not  the 
outcome  will  meet  the  requirements.  All  designs  that  meet  requirements  are  equally  good.  All  designs  that  fail  to 
meet  requirements  are  equally  bad.  Uncertainty  with  respect  to  meeting  requirements  is  managed  by  exception 
within  the  risk  management  process,  even  though  such  uncertainty  is  ubiquitous  with  regard  to  performance  and  cost 
requirements. 

Value-Centric  Design  chooses  the  best  design  whether  or  not  individual  attributes  exceed  a  threshold.  If  the  best 
design  is  not  good  enough,  the  system  is  presumably  not  ready  for  development.  Value-Centric  Design 
acknowledges  pervasive  uncertainty  and  manages  it,  both  to  exploit  opportunities  (upside  uncertainties)  as  well  as 
avoiding  less  desirable  prospects. 

The  next  few  sections  will  provide  more  detail  and  examples  of  Value-Centric  Design  in  action. 

III.  Value  Centric  Conceptual  and  Preliminary  Design 

In  Phase  1  of  the  F6  Program,  all  four  performer  teams  developed  Value-Centric  Design  Methodology  (VCDM) 
processes  to  direct  their  system  design  choices  in  conceptual  and  preliminary  design.  In  every  case,  the 
methodology  distinguished  design  alternatives  by  attributes,  typically  including  availability  and  always  including 
life  cycle  cost,  itself  a  collector  embracing  many  subsidiary  attributes  (for  example,  manufacturing  cost,  launch  cost, 
and  cost  of  operation).  A  system  value  model  was  constructed  to  evaluate  the  designs  based  on  the  attributes.  The 
methodologies  took  different  approaches  to  utilizing  these  evaluations  for  design  decisions. 

A.  Optimization 

The  broadest  implementation  of  VCDM  would  be  to  parameterize  the  design,  then  use  the  system  value  model  as 
an  objective  function  to  search  the  space  of  design  parameters  for  the  optimal  design.^  Automating  the  optimization 
process  is  not  as  easy  as  it  sounds  because  the  attributes  of  the  point  in  design  space  must  be  assessed  for  every  step 
in  the  search.  However,  some  contractors  developed  system  value  models  capable  of  projecting  attributes  from 
design  parameters  and  optimizing  the  design.  Even  without  this  feature,  optimization  could  be  done  with  engineers 
in  the  loop  estimating  attributes. 

Nevertheless,  the  performers  did  not  use  optimization  as  their  basic  design  process.  Instead,  engineering 
judgement,  backed  by  experience  on  conventional  spacecraft,  appeared  to  be  the  primary  guide  in  searching  for  the 
overall  system  design. 

B.  Trade  Studies 

Another  application  of  VCDM  is  to  discriminate  the  results  of  system  trade  studies.  Trade  alternatives  are 
constructed  and  the  attributes  of  each  alternative  are  quantified.  The  system  value  model  is  then  used  to  score  each 
alternative,  and  the  highest  scored  alternative  is  selected. 

Every  performer  team  used  VCDM  for  at  least  some  of  their  system  trade  studies.  Eor  example,  the  number  of 
modules  in  a  fractionated  cluster  and  the  heterogeneity  of  the  modules  were  used  to  construct  trade  studies  which 
determined  how  many  of  what  type  of  modules  would  be  included  in  the  eventual  operational  fractionated 
spacecraft.  (Maciuca  presents  such  an  example  in  Eig.  21.^) 

Often  other  criteria  entered  into  the  decision  in  addition  to  risk-adjusted  expectation  of  net  present  value.  This 
indicated  that  perhaps  not  all  the  important  attributes  were  present  in  the  system  value  model. 


*  The  traditional  systems  engineering  process  is  well  described  by  the  INCOSE*^  or  NASA**  Systems  Engineering 
Handbook.  The  US  Department  of  Defense  provides  similar  guidance  in  Chapter  4  of  the  Defense  Acquisition 
Guidebook,  an  online  dynamic  publication  (see  https://acc.dau.mil/dagch4). 

^  An  objective  function  is  the  mathematical  function  that  measures  goodness  during  an  optimization  search,  like  a 
score  that  shows  how  well  each  point  in  the  search  is  doing  relative  to  the  other.  It  is  much  like  the  childhood  search 
game  where  the  guide  says,  “You  are  getting  hotter... No,  now  you  are  colder... Now  hotter  than  ever!”  The 
objective  function  is  the  thermometer  that  guides  the  search. 
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C.  Rules  of  Thumb 

DARPA  asked  the  performers  to  apply  their  system  value  models  to  legacy  systems  and  thereby  try  to  gain  some 
insight  into  the  value  of  fractionation  and  the  ways  in  which  fractionation  might  be  most  effective.  From  these 
insights,  the  performers  developed  rules  to  serve  as  intuitive  guides  to  designing  the  F6  system. 

In  retrospect,  the  legacy  applications  may  have  been  too  few  to  draw  sound  generalizations  about  fractionation. 
Also,  the  performers  were  under  significant  time  and  budget  pressures,  which  perhaps  limited  the  depth  of  the 
analyses.  Finally,  it  might  be  that  the  final  stages  of  enlightenment  may  often  include  a  deja  vu  feeling  that  you 
knew  the  hard  sought  truth  all  along.  In  any  case,  the  rules  which  were  developed  felt  like  a  mix  of  common  sense, 
on  the  one  hand,  and  unsupported  overgeneralizations  on  the  other. 

We  may  need  to  wait  until  a  few  fractionated  systems  have  been  designed  to  a  greater  level  of  detail  before  we 
will  be  able  to  lay  down  some  solid,  useful  design  guidelines. 

D.  System  Value  Model 

The  major  element  in  every  performer’s  VCDM  effort  was  the  construction  of  a  system  value  model.  Each 
produced  an  elaborate  life  cycle  model,  some  with  automated  design  features  built  in,  which  could  work  from  a  set 
of  design  choices  and  environmental  variables  to  measures  of  life  cycle  cost  and  performance.  The  environmental 
variables  were  varied  probabilistically,  allowing  repeated  model  runs  to  generate  Monte  Carlo  simulations  of  value 
prospects.  Results  were  plotted  as  clouds  of  points  or  percentile  boundaries  on  the  spread  of  cases. 

All  the  models  were  capable  of  illuminating  trade  studies  by  indicating  the  higher  valued  alternatives  and 
providing  underlying  information  about  why  the  selected  alternative  was  better.  Phase  1  of  F6  answered  any  doubts 
about  the  practicality  of  value  models  for  use  in  the  design  of  space  systems. 

E.  Managing  Uncertainty 

The  most  powerful  benefits  of  fractionation  may  be  it  its  effectiveness  in  the  face  of  uncertainty.  Therefore,  the 
work  that  the  performer  teams  put  into  the  probabilistic  aspects  of  their  system  value  models  was  essential  to 
showing  the  true  value  of  F6.  Flexibility  and  robustness  are  terms  that  only  make  sense  in  an  uncertain  world,  and 
we  believe  that  responsive  space  is  important  because  the  environment  is  uncertain,  so  that  we  need  to  respond  to 
sudden  unexpected  threats  and  opportunities. 

Moreover,  uncertainty  is  an  essential  element  of  system  design,  particularly  in  the  conceptual  and  preliminary 
stages.  If  we  really  knew  how  a  design  would  turn  out,  engineering  would  simply  be  a  process  of  generating 
computer  aided  drawings  and  product  definitions.  A  design  is  proposed,  but  the  weight,  power  consumption, 
production  cost,  and  lifetime  are  all  uncertain.  Traditionally,  we  have  paid  little  attention  to  this  uncertainty  and 
assumed  the  design  would  realize  its  most  likely  weight,  cost,  performance,  and  so  on.  This  is  problematic,  even  on 
good  days,  because  the  distributions  of  these  attributes  tend  to  be  skewed  in  a  way  that  the  expectations  of  weight, 
cost,  power,  and  so  on,  are  significantly  worse  than  the  most  likely  forecasts. 

The  performer  value  models  addressed  uncertainty  by  using  Monte  Carlo  simulation  to  inject  on-orbit  failures, 
launch  failures,  shorter  and  longer  component  lifetimes,  funding  shortfalls,  schedule  delays,  and  all  manner  of  other 
variations.  The  Monte  Carlo  process  allows  very  unlikely  events,  and  randomly  distributed  occurrences,  such  as 
Weibull-characterized  component  failures,  to  be  incorporated  into  the  value  analysis  in  a  straightforward  and 
rigorously  correct  manner. 

However,  the  use  of  Monte  Carlo  simulation  creates  some  problems  of  its  own.  Optimization  is  difficult  when 
using  a  Monte-Carlo-simulated  objective  function,  for  two  reasons.  First,  Monte  Carlo  is  by  nature  not  repeatable. 
Two  runs  with  the  same  inputs  can  never  give  precisely  the  same  output,  even  with  tens  of  thousands  of  trials, 
because  the  environmental  variables  are  randomly  generated.  Optimization  gradient  search  techniques  and 
Newtonian  searches  rely  on  calculating  derivatives,  where  the  numerator  is  the  difference  in  the  value  of  two  very 
similar  inputs.  When  the  value  calculation  includes  a  Monte  Carlo  simulation,  most  of  the  difference  in  value  will 
be  due  to  random  variation  in  the  simulation  runs  rather  than  to  the  slight  difference  in  the  inputs.  (This  can  be 
avoided  by  preserving  the  randomized  environmental  variable  settings  for  all  ten  thousand  trials  and  reusing  exactly 
the  same  settings  for  the  second  inputs.  However,  this  becomes  more  of  a  designed  experiment  than  a  Monte  Carlo 
simulation.)  Second,  tens  of  thousands  of  Monte  Carlo  trials  can  take  a  long  time.  When  this  process  is  embedded 
in  the  inner  loop  of  an  optimization,  the  combination  can  be  so  slow  as  to  be  quite  inconvenient.  We  have  become 
unaccustomed  to  employing  design  tools  that  require  days  to  return  an  answer. 

In  the  future,  we  may  want  to  use  Monte  Carlo  to  characterize  the  impact  of  probabilistic  inputs,  but  then 
develop  a  smoother  and  faster  model,  perhaps  a  response  surface  model,  to  use  for  design  optimization. 
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F.  Summary  -  Conceptual  and  Preliminary  Design 

System  value  model  development  was  a  large  task  for  all  the  performers.  The  value  models  were  not  operational 
until  late  in  Phase  1,  after  many  of  the  design  decisions  were  made.  For  that  reason,  the  design  work  in  Phase  1  of 
F6  was  not  as  value-centric  as  it  might  have  been. 

The  takeaway  lesson  is  that  we  should  have  either  had  a  study  task  prior  to  starting  conceptual  design  in  which 
the  teams  focused  just  on  developing  value  models,  or,  more  realistically,  we  should  develop  a  much  simpler  and 
quicker  approach  to  building  small  elegant  value  models  for  conceptual  design  work.  When  Value-Centric  Design 
is  embraced  in  the  future  by  the  acquisition  community,  value  models  should  be  developed  in  the  Pre-Phase  A 
portion  of  a  program,  not  after.  We  have  not  yet  worked  out  what  descriptors  like  accuracy  and  realism  mean  in  the 
context  of  a  value  model,  but  these  models  are  clearly  much  different  from  models  of  physical  processes  like  heat 
transfer  or  fluid  dynamics.  As  we  gain  more  understanding  of  the  interaction  of  value  models  and  the  design 
process,  perhaps  we  will  get  a  better  handle  on  how  much  detail  is  enough  for  a  system  value  model.* 

IV.  Value- Centric  Risk  Management 

As  we  discussed  in  our  last  paper,*  Value-Centric  Design  opens  up  a  fundamentally  new  approach  to  risk 
management  within  system  development.  System  value  models  that  depict  uncertainty  and  express  value  in  dollars 
can  quantify  risks  with  clarity  and  precision  that  is  lacking  in  the  fever  charts  and  waterfalls  used  to  deal  with  risk 
today.  Also,  value  models  show  the  upside  as  well  as  the  downside  of  program  uncertainties,  whereas  traditional 
risk  management  focuses  entirely  on  the  downside,  resulting  in  an  overly  conservative  approach  to  advanced 
technology.  In  Phase  1,  the  F6  performer  teams  began  the  development  of  systems  engineering  risk  management 
strategies  based  on  Value-Centric  Design.  This  section  will  discuss  lessons  learned  from  that  effort  and  look  to  the 
way  ahead. 

A.  What  is  Risk? 

In  the  systems  engineering  sub-discipline  of  risk  management,  risk  is  the  possibility  that  something  undesirable 
may  occur.  In  contrast,  the  Value-Centric  approach  takes  the  view  of  economists,  that  a  risk  is  an  uncertainty  that  is 
relevant  to  a  current  decision  or  plan.  The  risk  describes  some  future  outcome  that  may  turn  out  well  or  badly  with 
some  distribution  of  probabilities.  On  the  good  side  are  opportunities,  or  upside  risks.  On  the  bad  side  are  the 
traditional  downside  risks. 

A  risk  management  system  will  need,  at  a  minimum,  to  address  technical  risk,  cost  risk,  and  program  risk. 

1.  Technical  risk 

A  technical  risk  is  a  particular  uncertainty  in  the  attributes  of  the  system  under  development,  usually  attached  to 
a  specific  component,  part,  technology,  or  design  that  is  the  source  of  the  uncertainty.  As  an  example,  a  new 
satellite  design  may  be  unsure  how  the  payload  can  be  mounted  on  the  deck  of  the  satellite  bus,  and,  depending  on 
the  mounting,  the  payload  may  interfere  with  a  standard  fairing  that  is  planned  for  the  launch  vehicle.  If,  as  the 
design  progresses,  the  interference  occurs,  then  a  custom  fairing  must  be  developed  or  the  bus  will  need  to  be 
modified. 

An  example  of  a  technology-based  technical  risk  might  be  a  new  polymer  battery  that  promises  to  weigh  less 
than  legacy  batteries.  However,  the  lifetime  of  the  new  battery  is  uncertain.  If  testing  reveals  that  the  battery 
lifetime  will  significantly  reduce  the  spacecraft  life,  the  system  may  have  lower  value  than  if  the  heavier  legacy 
battery  is  used.  Saleh**  provides  useful  data  of  the  impact  of  technology  readiness  level,  and  implicit  technology 
risk,  on  development  schedule. 

2.  Cost  risk 

A  cost  risk  is  a  specific  situation  that  causes  an  uncertainty  in  development  cost.  Consider  an  earth-observing 
spacecraft  development  program.  The  image  processing  software  will  be  developed  by  a  firm  in  India.  However, 
export  approval  for  the  software  requirements  specification  is  still  pending.  If  approval  is  denied,  a  contractor  in 
Los  Angeles  can  do  the  software,  but  it  will  be  much  more  expensive,  and  termination  costs  must  be  paid  to  the 
Indian  firm.  This  is  an  example  of  a  cost  risk. 

3.  Program  risk 

Program  risks  includes  risks  of  schedule  delays,  risks  to  meeting  necessary  milestones,  or  risks  that  arise  from 
dependencies  on  other  programs.  For  example,  a  spacecraft  might  be  designed  to  be  launched  by  a  rocket  that  is  still 


*  For  a  more  detailed  discussion  on  the  current  state  of  value  modeling  in  aerospace,  see  Collopy.*^ 
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under  development.  If  the  new  launch  system  does  not  successfully  enter  service,  the  spacecraft  will  need  to  be 
redesigned,  and  a  launch  slot  will  need  to  be  secured  on  another  launcher. 

To  evaluate  program  risks,  a  system  value  model  must  be  able  to  tie  a  monetary  value  to  program  delays.  This  is 
well  within  the  capability  of  the  value  models  developed  by  the  F6  performers. 

B.  Risk  and  decision  making 

Risks  can  only  be  managed  when  there  is  a  potential  present  or  future  action  that  can  be  taken  by  the  program  to 
address  the  uncertain  outcome.  For  example.  Congress  may  eliminate  the  budget  for  a  spacecraft  development 
program,  but,  unless  there  is  some  action  available  to  the  program  team  that  would  influence  Congress,  there  is 
nothing  the  team  can  manage  with  regard  to  the  risk.  Another  example  might  be  the  polymer  battery  case  described 
above.  What  if  the  program  commits  to  a  two-battery  power  system,  with  one  polymer  battery  and  one  legacy 
battery.  The  spacecraft  is  designed  for  the  worst  case  performance  of  the  polymer  battery.  The  design  decision  is 
made  and  there  is  no  turning  back.  This  ceases  to  be  a  risk  to  be  managed,  because,  even  though  the  outcome  is  still 
uncertain,  there  is  no  longer  a  decision  to  be  made.  The  future  will  simply  play  out.  In  another  case,  there  is 
uncertainty  whether  the  structure  of  a  vehicle  can  carry  necessary  torque  loads,  so  a  brace  is  added  to  increase  the 
load  capacity.  The  decision  is  complete,  and  there  is  no  longer  a  risk  to  manage. 

Uncertainty  is  the  essence  of  risk.  Consider  a  program  where  schedule  delays  threaten  a  planned  design  review. 
When  there  is  a  50%  chance  that  the  review  will  need  to  be  delayed,  this  situation  can  be  managed  as  a  risk.  Once 
there  is  a  95%  chance  the  review  will  need  to  be  delayed,  there  is  little  uncertainty  left.  It  is  time  to  reschedule  the 
review  and  drop  the  issue  from  the  risk  management  process. 

In  summary,  a  risk  needs  management  when  there  is  a  present  or  future  decision  to  be  made  in  the  face  of 
uncertainty.  Value-Centric  Design  can  place  a  dollar  value  on  the  prospects  that  may  occur,  and  aggregate  these 
into  an  expectation  of  value  for  each  alternative.  A  decision  analysis  can  be  performed  to  look  at  the  impact  of 
information  on  a  future  decision,  such  as  the  impact  of  results  from  a  verification  test.  How  likely  is  it  that  the  test 
data  will  change  the  decision?  Is  it  worthwhile  to  obtain  more  data  before  deciding? 

C.  Risk  Management  in  Systems  Engineering  Today 

In  the  status  quo  systems  engineering  process,  a  potential  problem  is  identified  as  a  risk,  a  mitigation  plan  is 
instituted,  and  the  plan  is  described  on  a  risk  waterfall  chart  where  every  action  reduces  the  expected  loss 
(probability  of  failure  times  consequence  of  failure)  of  the  risk  until  it  can  be  colored  green  on  a  risk-assessment 
fever  chart.  While  the  trip  down  the  waterfall  may  generate  a  sense  of  accomplishment,  and  even  drama,  it  is  hard 
to  understand  what  this  all  really  means.  If  a  plan  is  in  place  such  that,  when  it  is  complete,  there  will  be  little  or  no 
risk,  then  there  is  at  present  little  or  no  risk,  because  the  plan  is  in  place.  A  “waterfall”  chart  can  only  be  meaningful 
if  it  identifies  certain  future  points  where  information  will  arrive,  and  the  information  is  of  a  sort  that  can  make  the 
expected  outcome  better  or  worse.  That  is,  there  must  be  branches  in  the  chart,  and  at  each  branch,  some  water 
should  go  downhill  and  some  should  go  uphill.  Management  should  be  tracking  the  waterfall  because  there  will  be 
decisions  to  be  made  based  on  the  new  information. 

To  put  it  another  way,  how  can  a  risk  be  yellow  if,  no  matter  what  happens  in  the  future,  it  will  eventually  be 
green.  If  we  know  it  is  going  to  be  green  in  the  future,  then  it  is  green  now,  since  the  color  is  nothing  but  a 
projection  of  future  outcomes.  If  it  is  truly  yellow  today  and  may  become  green  in  the  future,  then  the  green 
prospect  must  be  balanced  by  a  red,  or  at  least  darker  yellow,  prospect  to  justify  the  “on  balance”  (or,  technically, 
“expected  value”)  rating  of  yellow. 

With  a  more  quantitative  approach  to  measuring  risk,  we  have  the  opportunity  to  implement  a  more  meaningful 
risk  management  program  that  focuses  program  management  on  real  decisions  and  the  importance  of  obtaining 
information  to  inform  those  decisions. 

D.  More  Work  to  Be  Done 

In  Phase  1,  the  F6  performer  teams  explored  the  evaluation  of  risks  such  as  launch  failures  using  their  system 
value  models.  Each  team  also  executed  a  classic  risk  management  process  as  part  of  Phase  1  systems  engineering 
work.  The  performers  have  laid  down  plans  to  integrate  Value-Centric  Design  and  their  risk  management  process 
during  the  next  phase  of  F6. 


V.  Value-Centric  Detailed  Design 

Looking  forward.  Phase  2  of  the  F6  program  will  extend  Value-Centric  Design  to  the  detailed  design  phase, 
where  it  will  be  applied  to  the  design  of  components  of  the  fractionated  spacecraft  modules.  How  this  is  done  will 
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be  up  to  the  performer,  but  some  possibilities  raised  in  previous  work  are  distributed  optimal  design**  and 
management  with  scoreboards.*®  These  approaches  were  not  introduced  in  Phase  1,  and  the  government  team  will 
not  be  requiring  them  in  Phase  2.  They  are  discussed  here  only  as  illustrations  of  the  possibilities  inherent  in  the 
Value-Centric  Design  perspective. 

A.  Distributed  Optimal  Design 

Optimal  design  literally  means  making  the  best  system,  and  who  would  not  want  the  best?  And  yet,  although 
optimization  is  now  widely  used  in  conceptual  design  studies,  very  little  optimization  is  done  in  the  detailed  design 
stage,  when  the  system  has  been  divided  up  into  components,  and  the  components  are  designed  by  independent 
teams.  In  part,  the  barrier  has  been  the  fear  that  components  would  be  suboptimized,  that  is,  improved  from  a 
perspective  that  conflicts  with  the  overall  system  view,  and  therefore  causes  more  hurt  than  help.  Value-Centric 
Design  offers  the  opportunity  to  create  objective  functions  for  each  component  that  are  consistent  with  the  system 
value  model.  When  design  choices  are  made  to  maximize  the  objective  function,  optimal  design  is  taking  place.  In 
this  way,  each  component  can  be  improved  from  a  perspective  that  is  deliberately  consistent  with  the  overall  system 
view,  so  that  suboptimization  is  no  longer  a  danger. 

Here  is  a  boiled-down  outline  of  how  component  objective  functions  can  be  derived  from  the  system  value 
model.  System  attributes  like  weight,  cost,  power  consumption,  and  reliability  are  aggregate  functions  of 
component  weights,  costs,  and  so  on.  If  the  individual  component  attributes  are  perturbed,  they  will  cause  system 
attributes  to  change,  and  therefore  will  change  system  value.  The  induced  change  in  system  value  divided  by  the 
amount  of  perturbation  in  the  component  attribute  can  be  used  as  the  coefficient  of  the  attribute  in  the  component 
objective  function.  By  perturbing  each  component  attribute,  one  at  a  time,  an  entire  component  objective  function 
can  be  derived,  as  illustrated  in  Figure  1  where  the  linear  objective  function  coefficients  are  placed  in  the  “gradient” 
column.  This  process  is  very  similar  to  sensitivity  analysis. 

If  every  component  is  designed  to  maximize  its  objective  function,  and  the  objective  functions  are  all  derived 
from  a  single  system  value  model  in  the  manner  just  described,  the  whole  system  is  optimized.  This  is  somewhat 
abstract,  but  is  easier  to  grasp  if  the  component  objective  functions  are  implemented  in  design  scoreboards. 


Figure  1.  Deriving  a  component  objective  function  by  perturbing  component  attributes.  A  single 
component  attribute  is  changed.  This  causes  system  attributes  to  change  which  changes  the  system  value.  The 
ratio  of  the  change  in  system  value  to  the  change  in  the  component  attribute  becomes  the  coefficient  of  the 
component  attribute  in  the  component  objective  function,  which  is  the  inner  product  of  the  status  column  and  the 
gradient  column  in  the  table. 
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B.  Design  Scoreboards 

Figure  2  shows  a  design  scoreboard  for  a  satellite  bus.  The  scoreboard  is  merely  a  linear  objective  function  laid 
out  in  table  form.  The  left  column  lists  the  names  of  the  component  attributes  which  describe  the  bus.  The  status 
column  shows  the  current  estimate  of  each  attribute.  These  attributes  are  the  input  to  the  objective  function.  The 
gradient  column  contains  the  coefficients  of  each  attribute  in  the  function.  The  value  column  is  the  product  of  the 
gradient  and  status,  computed  row  by  row.  The  Design  Value,  which  is  the  output  of  the  objective  function,  is  the 
sum  of  the  value  column  (and  thus  the  inner  product  of  the  gradient  and  status  columns). 

The  scoreboard  is  a  guide  for  design  decisions  and  optimization.  If  a  team  is  considering  an  alternative  design 
that  reduces  the  mass,  volume,  and  power  consumption  of  their  component  while  saving  cost,  that  is  an  easy 
decision.  The  alternative  is  better  because  it  is  better  in  every  way.  But  more  common  and  more  difficult  design 
choices  are  when  one  option  has  lower  mass  and  lower  cost  while  the  other  option  requires  less  power  and  volume 
and  increases  component  reliability.  Which  is  better?  If  the  attributes  of  each  option  are  plugged  into  the  Status 
column  in  the  design  scoreboard  and  the  Design  Value  is  computed  for  each,  the  better  option  will  show  itself 
because  it  will  have  more  design  value. 

The  scoreboard  is  a  natural  format  for  trade  studies.  The  study  option  with  the  greatest  design  value  is  the  one 
that  is  preferred.*  Systems  engineers  often  employ  trade  factors  in  doing  such  studies.  The  gradient  column  in  the 
scoreboard  implicitly  contains  all  the  trade  factors  between  the  attributes.  By  the  chain  rule  of  differentiation,  the 
trade  factors  are  simply  the  ratios  of  the  respective  attributes’  entries  in  the  column. 

C.  Expanding  the  Notion  of  Design  Optimization 

When  we  hear  the  term  “design  optimization,”  we  tend  to  think  of  elaborate  software  tools  that  perform 
automated  design.  However,  consider  a  component  design  team  working  with  a  scoreboard,  trying  out  design  after 
design  in  an  effort  to  improve  the  design  value.  The  next  design  they  choose  to  try  will  likely  be  very  similar  to  the 
previous  trials  that  have  shown  the  most  value. 

This  process  is  unequivocally  an  optimization  search,  where  the  scoreboard  is  the  objective  function.  However, 
it  is  optimization  by  hand,  or  optimization  with  engineers  in  the  loop.  Like  automated  optimization,  it  is  a  search  for 
the  very  best  design.  It  differs  from  optimization  driven  by  computerized  algorithms  because  it  is  slower,  and  it  is 
much  more  robust.  The  team  of  people  will  avoid  embarrassingly  bad  alternatives  that  a  software  tool  might  settle 
on.  And  the  humans  can  cope  with  very  poorly  structured  design  spaces  that  would  bedevil  an  automated  process. 

It  has  been  suggested  that  designing  components  to  allocated  requirements  is  the  chief  source  of  cost  and 
schedule  overruns  that  are  endemic  in  large  aerospace  and  defense  development  programs.^*  If  this  is  true,  the  use 
of  scoreboards  to  flow  direction  down  to  design  teams  is  an  alternative  to  allocated  requirements  that  avoids  the 
pitfalls  that  cause  overruns.  (This  method  is  in  fact  a  “guidance  technology”  in  the  nomenclature  of  Baldwin  and 


Status 

Gradient 

Value 

Mass 

Power  Capacity 

Propellant  Consumption 
Volume 

Manufacturing  Costs 
Development  Costs 
Reliability 

700  Kg 

2,450  Watts 

40  g  /  day 
4,000,000  cu.  cm. 
20,000,000  $ 
250,000,000  $ 

24,750  hrMTBF 

-30,000  $/ kg 

6,081  $/Watt 
-295,852  $  /  (gm/day) 
-2.00  $  /  cc 
-1.00  $/$ 

-0.02  $  /  $ 

21.94  $/ hrMTBF 

-21.00  $  millions 
14.90  $  millions 
-1 1.83  $  millions 
-8.00  $  millions 
-20.00  $  millions 
-5.02  $  millions 
0.54  $  millions 

Design  Value  -50.41  $  millions 

Figure  2.  Example  Scoreboard  for  a  Navigation  Satellite  Bus.  This  component  scoreboard  was  developed  by  the 
AIAA  Value-Driven  Design  Program  Committee  in  a  workshop  at  Orbital  Sciences  Corp.  in  April,  2006?^  It  has 
been  scaled  and  normalized  to  manufacturing  cost  for  readability. 


*  There  are  always  considerations  that  cannot  be  captured  in  a  scoreboard,  some  of  which  may  be  political  or 
otherwise  non-technical.  These  will  play  into  the  decision  and  may  dictate  the  outcome.  However,  the  scoreboard 
provides  a  good  starting  place  for  the  technical  side  of  a  decision. 
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Clark.^^)  But  the  complete  solution  to  reforming  large  system  design  will  necessitate  changes  to  the  way  large 
systems  are  acquired. 


VI.  Value-Centric  Design  in  Acquisition 

If  Value-Centric  Design  were  to  take  hold  in  mainstream  weapon  system  development  programs,  it  would  need 
to  be  at  least  endorsed  by,  and  more  likely  integrated  into,  the  defense  acquisition  system.  F6  is  an  aggressive 
technology  development  program  in  which  contractors  were  asked  to  develop  system  value  models  as  a  research 
activity  to  see  how  they  would  approach  the  task.  However,  on  a  full  development  program,  the  government  should 
develop  the  system  value  model  as  part  of  requirements  generation.  After  all,  it  is  the  government’s  value  of  the 
various  attributes  that  should  be  encoded  in  the  system  value  model,  not  the  contractor’s. 

A.  The  Current  State  of  Defense  Acquisition 

According  to  the  GAO,^^  current  weapon  systems  development  programs  are  overrunning  42%  in  development 
cost  and  25%  in  production  cost,  and  are  reaching  initial  operating  capability,  on  average,  22  months  behind 
schedule.  These  figures  are  not  inconsistent  with  Norman  Augustine’s  assessment  of  programs  in  the  late  1970’s 
and  early  1980’s,^‘*  which,  at  completion,  were  52%  over  in  combined  development  and  acquisition  cost  and  33% 
behind  in  schedule.  Augustine  used  only  completed  programs,  while  the  large  majority  of  the  GAO’s  sample  set  are 
still  in  development,  and  will  presumably  continue  to  get  later  and  more  expensive,  so  it  makes  sense  that 
Augustine’s  figures  are  higher. 

We  have  observed^*  that  the  requirements  allocation  process  naturally  brings  about  cost  growth  and  schedule 
delays  of  an  order  that  is  consistent  with  Augustine’s  observations.  The  effect  occurs  mainly  because  engineers  who 
are  asked  to  maximize  the  probability  that  a  component  will  meet  its  allocated  requirements  will  often  find  that  the 
safest  design  is  one  that  just  barely  meets  most  of  the  requirements.  The  result  is  a  marginal  system  that  needs 
several  redesigns  or  major  changes  to  achieve  functionality.  Every  redesign  increases  system  development  time  and 
cost,  and  most  redesigns  add  to  the  unit  production  cost. 

On  the  other  hand,  a  design  team  assigned  to  maximize  design  value  is  driven  to  choose  designs  that  far  exceed 
the  levels  of  typical  allocated  requirements.  The  result  is  a  robust  design  that  is  functional  or  nearly  functional  on 
the  first  go-round.  This  avoids  long  iterative  development  schedules  and  the  attendant  cost  growth. 

B.  Value-Based  Acquisition 

Carter  and  White^^  describe  how  Value-Centric  Design  would  be  integrated  into  the  acquisition  process.  They 
call  their  concept  Value-Based  Acquisition,  or  VBA.  To  quote; 

The  key  to  VBA  at  the  program  level  is  the  development  of  a  value  model  that  embodies  key  system  design  features, 
such  as  weight,  manufacturing  cost,  reliability,  and  the  like,  as  well  as  key  acquisition  concerns,  such  as  cost  and 
schedule  ...  Once  a  quantitative  value  model  has  been  defined,  it  can  become  the  basis  for  contracting.  A  program 
officer  can  offer  a  contract  in  which  price  is  a  function  of  value.  The  contract  would  specify  the  price  that  the 
government  would  be  willing  to  pay  for  different  levels  of  performance  . . .  Under  a  value-based  contract,  a  contractor 
maximizes  profit  by  including  only  those  features  whose  value  to  the  government  exceeds  their  cost. 

When  a  firm  accepts  a  contract  under  which  their  profit  is  directly  tied  to  a  system  value  model  evaluation  of 
their  current  design,  they  will  naturally  adopt  the  value  model  to  guide  the  design,  since  this  is  the  route  to 
maximizing  profits. 

The  firm  will  also  want  their  subcontractors  to  adopt  Value-Centric  Design,  to  enhance  profitability  and  to 
offload  risk  onto  the  subcontractors.  The  prime  contractor  will  be  driven  to  place  incentives  in  its  subcontracts  that 
directly  parallel  the  incentives  in  the  government’s  contract. 


During  the  Joint  Advanced  Strike  Technology  Program  in  1994,  a  precursor  to  the  Joint  Strike  Fighter 
development  program,  the  government  team  ran  a  tiered  set  of  simulations  (campaign-wide,  tactical,  mission,  etc.) 
and  used  the  data  to  build  response  surfaces  in  attribute  space,  all  in  the  name  of  requirements  generation.  These 
response  surfaces  contained  most  of  the  information  necessary  to  build  a  system  value  model  for  the  Joint  Strike 
Fighter. 
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VII.  Value- Centric  Design  Results  in  the  DARPA  F6  Program 

We  have  described  Value-Centric  Design  and  discussed  how  it  was  used  in  Phase  1  of  the  F6  Program.  We  have 
projected  ways  in  which  Value-Centric  Design  may  contribute  to  later  phases  of  system  development  programs.  In 
this  section,  we  will  relate  how  Value-Centric  Design  methods  specifically  contributed  to  the  development  of  the  F6 
demonstration  program  and  refinement  of  the  fractionated  spacecraft  concept. 

We  discuss  in  particular  the  models  and  simulations  developed  during  Phase  1  to  estimate  system  value  and  the 
frameworks  developed  for  design  optimization.  We  then  present  the  results  of  design  studies  on  fractionated 
spacecraft,  which  were  conducted  using  these  tools. 

A.  System  Value  Models 

Each  of  the  F6  performer  teams  developed  a  system  value  model  to  estimate  the  risk-adjusted  net  present  value 
of  putative  satellite  configurations  in  order  to  search  for  the  best  application  of  fractionation.  “Net”  implies  benefit 
minus  cost,  so  it  is  natural  to  consider  these  models  in  three  parts:  Cost  models,  benefit  models,  and  the  estimation 
and  evaluation  of  risk. 

7.  Cost  Models 

Highly  sophisticated  cost  models  have  been  available  for  a  long  time,  particularly  in  the  satellite  industry.^^’^^ 
However,  the  F6  teams  developed  models  which  have  achieved  new  levels  of  complexity  and  power.  Along  with 
parametric  estimators  of  component  costs,  the  models  employ  discrete  event  simulators  to  account  for  program 
events,  such  as  launch  failures  and  development  delays,  and  on-orbit  component  failures.  Each  of  the  performer 
teams  incorporated  probabilistic  simulation  into  their  cost  models,  so  that  the  resulting  cost  estimates  are  not  point 
estimates,  but  instead  probability  distributions  of  estimates.  Probabilistic  estimation  is  essential  to  cost  accuracy 
(see  Hazelrigg,*°  p.  270  ff.),  so  this  is  a  positive  feature. 

The  models  included  component-level  manufacturing  cost  models,  development  cost  models,  launch  cost 
estimation,  and  post-launch  system  operation  cost  models. 

2.  Benefit  Models 

While  cost  modeling  is  old  hat,  benefit  modeling  is  more  unique  to  the  E6  program,  and  therefore  could  present  a 
greater  challenge.  All  of  the  performer  teams  anchored  their  benefit  models  on  a  common  measure:  the  amount  of 
time  that  the  space  system  was  up  and  operating,  feeding  data  to  the  ground. 

The  Orbital  Sciences  team"*  developed  a  sophisticated  model  for  pricing  the  data  feed,  based  on  market 
dynamics.  To  develop  this  model,  they  used  the  same  logic  and  company  resources  that  go  into  creating  a  business 
plan  for  a  commercial  satellite.  The  Lockheed  Martin  team^  established  a  constant  price  per  megabyte  for  the  data 
feed.  Boeing*  based  the  price  of  data  on  a  conservative  estimate  of  system  cost,*  such  that  the  system  was  ensured 
of  attaining  a  reasonable  profit  margin. 

The  Northrop  Grumman^  team  used  Multi-Attribute  Utility  Theory^*  to  estimate  the  value  of  attributes  of  the 
service  provide  by  their  satellite  system.  Each  attribute  was  valued  on  a  dimensionless  utility  scale,  then  the 
attribute  utilities  were  merged  into  a  single  summary  utility. 

3.  Risk  Evaluation 

The  designers  used  Monte  Carlo  simulation  to  manifest  the  uncertainty  in  system  designs  and  configurations. 
Risk  was  then  quantified  as  the  variance  or  standard  deviation  in  overall  cost  and  benefit.  Lockheed  Martin 
provided  plots  of  the  expected  net  present  value  of  alternatives  versus  the  standard  deviation  of  net  present  value, 
which  illustrate  the  risk  return  frontier  of  best  options.  Orbital  plotted  net  present  value  versus  cost,  which  allowed  a 
quick  visual  assessment  of  the  impact  of  budget  constraints.  The  scatter  of  the  cloud  of  Monte  Carlo  results 
indicates  the  amount  of  risk.  Boeing  plotted  benefit  versus  cost,  so  that  the  benefit/cost  ratio  becomes  the  slope  of  a 
line  between  the  design  point  and  the  origin.  Again,  scatter  indicated  risk.  Boeing  fit  the  points  to  a  normal 
distribution  to  estimate  lo,  2  o,  and  3  O  standard  deviation  intervals  and  plotted  ellipses  around  the  mean  of  the  data 
points  to  roughly  indicated  the  spread.  Northrop  Grumman  plotted  dimensionless  utility  versus  cost. 

We  still  have  some  work  to  do  solidifying  the  concept  of  risk-adjusted  net  present  value,  which  is  a  candidate  for 
our  summary  evaluator  of  E6  designs.  Boeing  used  the  3o  lower  limit  of  benefit/cost  ratio  as  risk  adjusted  measure. 
The  other  teams  did  not  integrate  risk,  benefit,  and  cost  into  a  single  measure. 

In  some  of  the  early  work  that  led  up  to  Value-Centric  Design,  Saleh^^  explored  the  application  of  real  options 
theory,  including  the  Black-Scholes  equation,  to  quantify  the  impact  of  uncertain  environmental  factors  on  a  space 
system  that  could  respond  to  uncertainty  in  a  flexible  manner.  Interestingly,  although  all  the  performer  teams  were 
aware  of  this  approach,  none  employed  either  the  real  options  value  lattice  or  the  Black-Scholes  equation  in  their 
system  value  models.  Instead,  the  discrete  event  simulators  implicitly  implemented  something  like  the  method  that 
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Saleh  describes  as  decision  tree  analysis,  in  which  the  program  takes  decisions  based  on  events  that  have  occurred 
and  the  probability  of  events  that  may  occur.  It  may  be  that  the  Black-Scholes  approximation  is  not  as  useful  when 
the  system  is  already  represented  by  a  fairly  detailed  discrete  event  simulation. 

B.  Optimization 

Some  of  the  performer  teams  incorporated  their  system  value  model  into  an  automated  value-centric  design  tool. 
These  tools  were  made  up  of  a  design  generator  and  an  optimizer  which  used  the  system  value  model  as  the 
objective  function. 

7.  Generator 

Boeing  parameterized  the  design  space  and  generated  a  factorial  set  of  designs  spread  around  the  space. 
Lockheed  Martin  used  MIT’s  Generalized  Information  Network  Analysis^®  tool  to  generate  fractionated  spacecraft 
configurations.  The  Orbital  Sciences  team  employed  Georgia  Tech’s  GT-FAST  automated  design  tool  to  generate 
satellite  configurations  and  component  designs. 

2.  Optimizer 

Lockheed  Martin  used  the  MIT  Space  Lab’s  Time-Expanded  Decision  Network^*  optimization  framework, 
which  specifically  addresses  system  flexibility  and  adaptability  in  a  multi-level  optimization.  Orbital  Sciences 
developed  a  custom  optimizer,  the  PIVOT  tool,  described  in  Figure  1  of  their  paper.**  Boeing’s  approach  was  to 
visually  pick  out  the  best  design  on  a  plot  of  risk-adjusted  benefits  and  costs. 

C.  Value  of  Fractionation 

The  F6  Value-Centric  Design  tools  are  of  interest  in  themselves,  but  the  F6  Program  is  particularly  focused  on 
the  results  of  applying  these  tools  to  spacecraft  designs  that  use  fractionation.  While  we  may  expect  in  the  future  to 
find  systems  that  are  only  possible  with  fractionated  architectures.  Phase  1  of  F6  began  simply  by  looking  at 
fractionated  alternative  configurations  to  existing  or  near  term  space  systems.  Therefore,  the  essence  of  these  design 
studies  was  the  exploration  of  the  questions  “Does  fractionation  provide  a  better  design?”  and  “If  so,  what  about 
fractionation  makes  the  design  better?”  Through  these  studies,  we  have  gained  insight  into  what  is  important  and 
useful  about  spacecraft  fractionation. 

1.  Earlier  Studies 

Some  early  clues  were  provided  by  studies  that  led  up  to  the  F6  program.  Saleh^®  studied  on-orbit  servicing,  a 
strategy  that  shares  an  important  feature  with  fractionation:  when  an  operational  spacecraft  ceases  to  function,  it  can 
be  brought  back  into  operation  without  launching  a  full  replacement  satellite.  Saleh  concluded  that  on-orbit 
servicing  can  provide  net  value,  but  this  will  not  be  apparent  in  a  deterministic  comparative  cost  analysis.  First, 
benefits  must  be  estimated  along  with  costs,  because  increased  benefits  of  on-orbit  servicing  account  for  substantial 
value.  Cost  effectiveness  analyses  alone  would  not  show  the  benefit  of  these  novel  strategies.  Second,  a 
probabilistic  analysis  is  essential  because  much  of  the  value  of  on-orbit  servicing  is  derived  from  the  flexibility  it 
provides. 

Mathieu  and  Weigel^^  directly  examined  the  value  of  fractionation,  focusing  on  two  applications:  a 
communication  satellite  system  and  a  navigation  satellite  system.  In  both  applications,  two  fractionated 
configurations  showed  significantly  lower  life  cycle  cost  than  a  monolithic  satellite.  In  the  first  low  cost 
configuration,  one  module  contained  the  payload  and  a  second  module  contained  communication,  data  handling,  and 
computation  functions,  with  wireless  communications  between  the  modules.  The  second  low  cost  configuration  was 
like  the  first  except  that  communication  was  separated  onto  its  own  module,  for  a  total  of  three  modules.  Several 
configurations  with  larger  numbers  of  modules  were  predicted  to  have  cost  comparable  to  a  traditional  monolithic 
configuration,  with  lower  cost  if  high  unit  volume  manufacturing  savings  (steep  learning  curve)  was  achieved,  but 
higher  cost  than  the  monolith  if  volume  effects  were  not  as  strong.  Meanwhile,  all  of  the  fractionated  configurations 
delivered  greater  utility  than  the  monolithic  configurations,  so  in  the  end  the  net  value  of  fractionation  may  be  quite 
large  depending  on  the  equivalent  monetary  value  of  the  utility  measures. 

2.  Is  Fractionation  Better? 

F6  Phase  1  performers  reported  studies  showing  benefits  to  fractionated  spacecraft  over  traditional  monolithic 
satellites,  confirming  the  earlier  analyses.  The  Boeing  team  showed  that  an  eight  module  fractionated  configuration 
had  higher  cost,  but  higher  benefit,  and  a  higher  benefit/cost  ratio.*  The  fractionated  spacecraft  also  had  much  lower 
risk,  indicated  by  a  much  tighter  spread  of  cost  and  benefit  estimates.  The  study  notes  that,  while  configurations 
with  more  modules  (more  highly  fractionated)  showed  greater  benefit/cost  ratios,  the  really  steep  jumps  in 
benefit/cost  ratio  occur  going  from  a  monolithic  spacecraft  to  configurations  with  two  or  three  modules. 

Lockheed  Martin  found  many  fractionated  configurations  with  10%  to  20%  greater  expected  net  present  value 
than  an  equivalent  monolithic  satellite.  All  of  these  higher  valued  configurations  used  two  or  three  modules — 
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configurations  with  more  than  three  modules  had  less  value  than  the  monolith.  About  half  of  the  higher  valued 
configurations  also  had  significantly  lower  risk  than  the  monolith,  where  risk  was  measured  as  the  standard 
deviation  of  value. 

5.  Why  is  Fractionation  Better? 

Perhaps  more  important  than  the  absolute  results  about  which  configurations  have  higher  value  is  the  insight  into 
what  properties  make  good  fractionated  designs.  How  can  the  concept  of  fractionation  best  be  exploited? 

Mentioned  already  is  a  common  finding  that  two  or  three  module  configurations  show  the  most  dramatic 
increase  in  value  when  functionally  replacing  a  monolithic  satellite  design.  However,  this  rule  would  not 
necessarily  apply  to  fractional  architectures  for  wholly  new  capabilities.  Mathieu  and  Weigel^^  note  that 
fractionation  provides  scalability,  which  implies  that  very  high  performance  space  systems  might  employ  many, 
many  modules. 

Two  of  the  performer  teams  noted  the  importance  of  component  technology  readiness  level  (TRL)  to  module 
configuration.  The  Boeing  team  found  that  isolating  a  low  TRL  (technologically  immature)  component  on  a  module 
more  or  less  by  itself  improved  overall  system  value.*  In  a  similar  vein,  Lockheed  Martin  discovered  it  is  a  good 
practice  to  put  similar  TRL  components  together  on  a  module,  that  is,  to  segregate  components  by  TRL  among  the 
modules.^ 

Perhaps  most  significantly,  fractionated  spacecraft  show  less  uncertainty  in  value,  that  is,  less  risk,  when  faced 
with  random  environmental  events  and  random  changes  in  needs  and  requirements.  These  are  precisely  the  benefits 
of  robustness  and  flexibility  that  we  were  anticipating. 

VIII.  Conclusion 

Phase  1  of  the  F6  Program  was  a  challenging  venture  into  the  unexplored  territory  of  fractionated  spacecraft 
design.  Through  intense  efforts  from  the  four  performer  teams,  a  great  deal  was  learned  about  fractionation,  and  a 
wealth  of  experience  was  accumulated  in  the  new  systems  engineering  processes  of  Value-Centric  Design.  Value- 
Centric  Design  was  implemented  by  each  of  the  teams,  and  it  successfully  contributed  to  the  fractionated  spacecraft 
designs  which  they  developed.  We  can  expect  even  better  results  from  Value-Centric  Design  in  the  future  as  its 
methods,  processes  and  tools  become  more  mature. 

In  Phase  2,  we  will  continue  to  develop  and  apply  Value-Centric  Design  methodology.  We  will  execute  the 
detailed  design  of  F6,  which  will  introduce  additional  methodological  challenges. 

Some  of  the  areas  we  expect  to  delve  more  deeply  into  are 

•  Risk  management 

•  Value-based  acquisition  and  subcontracting 

•  Dynamic  project  management 

•  Benefit  assessment 

We  look  forward  to  reporting  the  results  at  the  end  of  Phase  2. 
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